{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "06d45bcc-8e5d-4f6d-9135-5fd4bb0033bd",
   "metadata": {},
   "source": [
    "# Quickstart: Deploy an AI Application with Seldon Core 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a51aeb0-3d8c-4126-b765-973d9104df22",
   "metadata": {},
   "source": [
    "In this notebook, we will demonstrate how to deploy a production-ready AI application with Seldon Core 2. This application will have two components - an sklearn model and a preprocessor written in Python - leveraging Core 2 **Pipelines** to connect the two. Once deployed, users will have an endpoint available to call the deployed application. The inference logic can be visualized with the following diagram:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9cfa396a-8b8a-49b3-94d3-ff3400e72644",
   "metadata": {},
   "source": [
    "```mermaid\n",
    "    graph LR\n",
    "        I[(Input)] --> PP\n",
    "        subgraph Pipeline\n",
    "            PP([Preprocessor]) --> M([Model])\n",
    "        end\n",
    "        M --> O[(Output)]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fcb013c-d5c4-4571-a5aa-a71246c9b3e4",
   "metadata": {},
   "source": [
    "To do this we will:\n",
    "\n",
    "1. Set up a **Server** resource to deploy our models\n",
    "2. Deploy an sklearn **Model**\n",
    "3. Deploy a multi-step **Pipeline**, including a preprocessing step that will be run before calling our model.\n",
    "5. Call our inference endpoint, and observe data within our pipeline\n",
    "\n",
    "{% hint style=\"info\" %}\n",
    "**Setup**: In order to run this demo, you need to connect to a cluster set up with an installation of Core 2 (see [here](../../installation/README.md)). We will be using the `kubectl` command line tool to interact with the Kubernetes cluster's control plane. Lastly, we will be using the `gcloud` CLI to pull models from Seldon's cloud storage, where we provide sample models and files. Once you are set up, you can run this demo as a pre-built jupyter notebook by accessing it in our github repo (the v2 branch), under `docs-gb/getting-started/quickstart/quickstart.ipynb`\n",
    "{% endhint %}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a3e0fae-2b9c-4aff-b3f3-83abacd1949c",
   "metadata": {},
   "source": [
    "## Step 1: Deploy a Custom Server"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d478e7a-6e2f-4275-a50d-c663f45883c6",
   "metadata": {},
   "source": [
    "As part of the Core 2 installation, you will have install MLServer and Triton Servers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "9b38c7f7-afd2-4d5c-bb06-8b73f13cc253",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME              READY   REPLICAS   LOADED MODELS   AGE\n",
      "mlserver          True    1          0               156d\n",
      "mlserver-custom   True    1          0               38d\n",
      "triton            True    1          0               156d\n"
     ]
    }
   ],
   "source": [
    "!kubectl get servers -n seldon-mesh"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7dfb1d89-42cd-449d-93fb-fcc1b4650f94",
   "metadata": {},
   "source": [
    "The server resource outlines attributes (dependency requirements, underlying infrastrucuture) for the runtimes that the models you deploy will run on. By default, MLServer supports the following frameworks out of the box: `alibi-detect`, `alibi-explain`, `huggingface`, `lightgbm`, `mlflow`, `python`, `sklearn`, `spark-mlib`, `xgboost`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62d60106-2963-444d-8ea9-d4c46a636264",
   "metadata": {},
   "source": [
    "In this example, we will create a new custom MLServer that we will tag with `income-classifier-deps` under **capabilities** (see docs [here](../../servers/servers.md#custom-capabilities)) in order to define which Models will be matched to this Server. In this example, we will deploy both our model (`sklearn`) and our preproccesor (`python`) on the same Server. This is done using the manifest below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ced3d40e-7817-4f58-bc5a-617d8bd7154e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "apiVersion: mlops.seldon.io/v1alpha1\n",
      "kind: Server\n",
      "metadata:\n",
      "  name: mlserver-custom\n",
      "spec:\n",
      "  serverConfig: mlserver\n",
      "  capabilities:\n",
      "  - income-classifier-deps\n",
      "  podSpec:\n",
      "    containers:\n",
      "    - image: seldonio/mlserver:1.6.0\n",
      "      name: mlserver\n"
     ]
    }
   ],
   "source": [
    "!cat  ../../../samples/quickstart/servers/mlserver-custom.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0b8feb36-d54a-4dd9-a40e-d9dddd211340",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "server.mlops.seldon.io/mlserver-custom unchanged\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply  -f ../../../samples/quickstart/servers/mlserver-custom.yaml -n seldon-mesh"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47972844-b409-4b8c-8a82-b877021fffce",
   "metadata": {},
   "source": [
    "## Step 2: Deploy Models"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb1afc21-2cbe-4d88-962b-d294e17a4cc7",
   "metadata": {},
   "source": [
    "Now we will deploy a model - in this case, we are deploying a categorical model that has been trained to take 12 features as input, and output **[0]** or **[1]**, representing a **[Yes]** or **[No]** prediction of whether or not an adult with certain values for the 12 features is making more than $50K/yr. This model was trained using the Census Income (or \"Adult\") Dataset. Extraction was done by Barry Becker from the 1994 Census database. See [here](https://archive.ics.uci.edu/dataset/20/census+income) for more details. \n",
    "\n",
    "The model artefact is currently stored in Seldon's a Google bucket - the contents of the relevant folder are below. Alongside our model artefact, we have a `model-settings.json` file to help locate and load the model. For more information on the Inference artefacts we support and how to configure them, see [here](https://docs.seldon.ai/seldon-core-2/user-guide/models/inference-artifacts)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "c99fcdd9-a4a2-46dd-9119-0d9b7c035bc3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gs://seldon-models/scv2/samples/mlserver_1.4.0/income-sklearn/classifier/:\n",
      "gs://seldon-models/scv2/samples/mlserver_1.4.0/income-sklearn/classifier/model-settings.json\n",
      "gs://seldon-models/scv2/samples/mlserver_1.4.0/income-sklearn/classifier/model.joblib\n",
      "\n",
      "\n",
      "Updates are available for some Google Cloud CLI components.  To install them,\n",
      "please run:\n",
      "  $ gcloud components update\n",
      "\n",
      "\n",
      "\n",
      "To take a quick anonymous survey, run:\n",
      "  $ gcloud survey\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!gcloud storage ls --recursive gs://seldon-models/scv2/samples/mlserver_1.4.0/income-sklearn/classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "36b7c9b8-48ad-4dba-9eaa-35d041b69c71",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"name\": \"income\",\n",
      "    \"implementation\": \"mlserver_sklearn.SKLearnModel\",\n",
      "    \"parameters\": {\n",
      "        \"uri\": \"./model.joblib\",\n",
      "        \"version\": \"v0.1.0\"\n",
      "    }\n",
      "}"
     ]
    }
   ],
   "source": [
    "!gsutil cat gs://seldon-models/scv2/samples/mlserver_1.4.0/income-sklearn/classifier/model-settings.json "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "955476a4-b7d4-44c3-924b-a33c747aa1d3",
   "metadata": {},
   "source": [
    "In our **Model** manifest below, we point to the location of the model artefact using the `storageUri` field. You will also notice that we have defined `income-classifier-deps` under **requirements**. This will match the Model to the Server we deployed above, as Models will only be deployed onto Servers that have **capabilities** that match the appropriate requirements defined in the Model manifest."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "92d75f55-0f23-4e0f-a295-8c60cb914433",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "apiVersion: mlops.seldon.io/v1alpha1\n",
      "kind: Model\n",
      "metadata:\n",
      "  name: income-classifier\n",
      "spec:\n",
      "  storageUri: \"gs://seldon-models/scv2/samples/mlserver_1.4.0/income-sklearn/classifier\"\n",
      "  requirements:\n",
      "  - income-classifier-deps\n",
      "  memory: 100Ki\n"
     ]
    }
   ],
   "source": [
    "!cat ../../../samples/quickstart/models/sklearn-income-classifier.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef93f7b0-cc68-4f43-8840-e8e8bc5d2a5a",
   "metadata": {},
   "source": [
    "In order to deploy the model, we will apply the manifest to our cluster:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "733610f4-0c31-4993-b9e9-4428a23a2451",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "model.mlops.seldon.io/income-classifier created\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f ../../../samples/quickstart/models/sklearn-income-classifier.yaml -n seldon-mesh"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "107bebfc-1772-4cd3-9012-39702ceef72e",
   "metadata": {},
   "source": [
    "We now have a deployed model, with an associated endpoint."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "416c0210",
   "metadata": {},
   "source": [
    "### Make Requests\n",
    "The endpoint that has been exposed by the above deployment will use an IP from our service mesh that we can obtain as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c387dac8-87ca-48a6-b590-9d3368182d9d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'34.32.149.48'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MESH_IP = !kubectl get svc seldon-mesh -n seldon-mesh -o jsonpath='{.status.loadBalancer.ingress[0].ip}'\n",
    "MESH_IP = MESH_IP[0]\n",
    "MESH_IP"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "13c34c1a-8379-46a9-a3ac-f4df17436abc",
   "metadata": {},
   "outputs": [],
   "source": [
    "endpoint = f\"http://{MESH_IP}/v2/models/income-classifier/infer\"\n",
    "headers = {\n",
    "    \"Seldon-Model\": \"income-classifier\",\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fac0628f-d107-49f4-9c95-997b91992da7",
   "metadata": {},
   "source": [
    "Requests are made using the **Open Inference Protocol**. More details on this specification can be found in our [docs](https://docs.seldon.ai/seldon-core-2/user-guide/inference/v2), or in the API documentation generated by our protocol buffers in the case of gRPC usage [here](https://buf.build/seldon/open-inference-protocol/docs/main:inference). This protocol is also supported by shared by Triton Inference Server for serving Deep Learning models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "73857842-cc9c-44c7-a8b2-2ec2a22fc045",
   "metadata": {},
   "outputs": [],
   "source": [
    "inference_request = {\n",
    "  \"inputs\": [\n",
    "    {\n",
    "      \"name\": \"income\",\n",
    "      \"datatype\": \"INT64\",\n",
    "      \"shape\": [1, 12],\n",
    "      \"data\": [53, 4, 0, 2, 8, 4, 2, 0, 0, 0, 60, 9]\n",
    "    }\n",
    "  ]\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b0b5af6-0bef-46ac-9fa0-084175ab5709",
   "metadata": {},
   "source": [
    "We are now ready to send a request!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d871e14f-18ae-4c95-99e4-f1ff0ec27f27",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'model_name': 'income-classifier_1',\n",
       " 'model_version': '1',\n",
       " 'id': '626ebe8e-bc95-433f-8f5f-ef296625622a',\n",
       " 'parameters': {},\n",
       " 'outputs': [{'name': 'predict',\n",
       "   'shape': [1, 1],\n",
       "   'datatype': 'INT64',\n",
       "   'parameters': {'content_type': 'np'},\n",
       "   'data': [0]}]}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import requests\n",
    "response = requests.post(endpoint, headers=headers, json=inference_request)\n",
    "response.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d7ff4b2-871f-4e5a-8a01-2ff55c3dcc11",
   "metadata": {},
   "source": [
    "We can see above that the model returned a `'data': [0]` in the output. This is the prediction of the model, indicating that an individual with the attributes provided is most likely making more than $50K/yr."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "025f4516-7cdc-493c-b760-8aa5d96c782b",
   "metadata": {},
   "source": [
    "## Step 3: Create and Deploy a 2-step Pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc22cc03-6f2a-49a3-97eb-7ff793b005cb",
   "metadata": {},
   "source": [
    "Often we'd like to deploy AI applications that are more complex than just an individual model. For example, around our model we could consider deploying pre or post-processing steps, custom logic, other ML models, or drift and outlier detectors. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "223563e4-adf6-41d3-afe8-1c0e0d9a98fa",
   "metadata": {},
   "source": [
    "### Deploy a Preprocessing step written in Python"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79aa97fd-63d3-4855-90fe-c2d03739d5e9",
   "metadata": {},
   "source": [
    "In this example, we will create a preprocessing step that extracts numerical values from a text file for the model to use as input. This will be implemented with custom logic using Python, and deployed as **custom model with MLServer**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "65a9e14c-e187-4b62-a163-ff124ff6bbd6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "import numpy as np\n",
    "\n",
    "# Extracts numerical values from a formatted text and outputs a vector of numerical values.\n",
    "def extract_numerical_values(input_text):\n",
    "\n",
    "    # Find key-value pairs in text\n",
    "    pattern = r'\"[^\"]+\":\\s*\"([^\"]+)\"'\n",
    "    matches = re.findall(pattern, input_text)\n",
    "    \n",
    "    # Extract numerical values\n",
    "    numerical_values = []\n",
    "    for value in matches:\n",
    "        cleaned_value = value.replace(\",\", \"\")\n",
    "        if cleaned_value.isdigit():  # Integer\n",
    "            numerical_values.append(int(cleaned_value))\n",
    "        else:\n",
    "            try:  \n",
    "                numerical_values.append(float(cleaned_value))\n",
    "            except ValueError:\n",
    "                pass  \n",
    "    \n",
    "    # Return array of numerical values\n",
    "    return np.array(numerical_values)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e111f09-e732-4ed3-9173-81e8a317446c",
   "metadata": {},
   "source": [
    "Before deploying the preprocessing step with Core 2, we will test it locally:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "4d86cce9-5d65-42c4-8da2-04008ebf6573",
   "metadata": {},
   "outputs": [],
   "source": [
    "input_text = '''\n",
    "\"Age\": \"47\",\n",
    "\"Workclass\": \"4\",\n",
    "\"Education\": \"1\",\n",
    "\"Marital Status\": \"1\",\n",
    "\"Occupation\": \"1\",\n",
    "\"Relationship\": \"0\",\n",
    "\"Race\": \"4\",\n",
    "\"Sex\": \"1\",\n",
    "\"Capital Gain\": \"0\",\n",
    "\"Capital Loss\": \"0\",\n",
    "\"Hours per week\": \"68\",\n",
    "\"Country\": \"9\",\n",
    "\"Name\": \"John Doe\"\n",
    "'''\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "2b952eec-77ae-4464-940a-fa0ae8142338",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([47,  4,  1,  1,  1,  0,  4,  1,  0,  0, 68,  9])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "extract_numerical_values(input_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b8ac78b-2d7c-450c-87ad-c4571beb576a",
   "metadata": {},
   "source": [
    "Now that we've tested the python script locally, we will deploy the preprocessing step as a **Model**. This will allow us to connect it to our sklearn model using a Seldon **Pipeline**. To do so, we store in our cloud storage an inference artefact (in this case, our Python script) alongside a `model-settings.json` file, similar to the model deployed above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "cca62aa6-e2d3-417e-ab44-9be66d12b055",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gs://seldon-models/scv2/samples/preprocessor/:\n",
      "gs://seldon-models/scv2/samples/preprocessor/model-settings.json\n",
      "gs://seldon-models/scv2/samples/preprocessor/model.py\n",
      "gs://seldon-models/scv2/samples/preprocessor/preprocessor.yaml\n"
     ]
    }
   ],
   "source": [
    "!gcloud storage ls --recursive gs://seldon-models/scv2/samples/preprocessor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "c40d1afd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "apiVersion: mlops.seldon.io/v1alpha1\n",
      "kind: Model\n",
      "metadata:\n",
      "    name: preprocessor\n",
      "spec:\n",
      "    storageUri: \"gs://seldon-models/scv2/samples/preprocessor\"\n",
      "    requirements:\n",
      "    - income-classifier-deps\n"
     ]
    }
   ],
   "source": [
    "!cat ../../../samples/quickstart/models/preprocessor/preprocessor.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10e7af2e-8669-4365-b5db-2da6de6366b9",
   "metadata": {},
   "source": [
    "As with the ML model deployed above, we have defined `income-classifier-deps` under **requirements**. This means that both the preprocesser and the model will be deployed using the same **Server**, enabling consolidation in terms of the resources and overheads used (for more about Multi-Model Serving, see [here](https://docs.seldon.ai/seldon-core-2/user-guide/models/mms))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "81702fa0-fe5e-409e-a208-b06657e8f429",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "model.mlops.seldon.io/preprocessor created\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f ../../../samples/quickstart/models/preprocessor/preprocessor.yaml -n seldon-mesh"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7131412-aae7-4014-aa67-cf43134a4bbe",
   "metadata": {},
   "source": [
    "We've now deployed the prepocessing step! Let's test it out by calling the endpoint for it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "0c67cee1-de7a-4fec-9985-08194a171e97",
   "metadata": {},
   "outputs": [],
   "source": [
    "endpoint_pp = f\"http://{MESH_IP}/v2/models/preprocessor/infer\"\n",
    "headers_pp = {\n",
    "    \"Seldon-Model\": \"preprocessor\",\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "bcaae6d0-5126-4eb0-a0ec-734f99a6dc6f",
   "metadata": {},
   "outputs": [],
   "source": [
    "text_inference_request = {\n",
    "    \"inputs\": [\n",
    "        {\n",
    "            \"name\": \"text_input\", \n",
    "            \"shape\": [1], \n",
    "            \"datatype\": \"BYTES\", \n",
    "            \"data\": [input_text]\n",
    "        }\n",
    "    ]\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "4de07b25-4272-404d-9e3d-0c5c5558aebd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'model_name': 'preprocessor_1',\n",
       " 'model_version': '1',\n",
       " 'id': 'b26e49d5-2a4c-488b-8dff-0df850fbed3d',\n",
       " 'parameters': {},\n",
       " 'outputs': [{'name': 'output',\n",
       "   'shape': [1, 12],\n",
       "   'datatype': 'INT64',\n",
       "   'parameters': {'content_type': 'np'},\n",
       "   'data': [47, 4, 1, 1, 1, 0, 4, 1, 0, 0, 68, 9]}]}"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import requests\n",
    "response = requests.post(endpoint_pp, headers=headers_pp, json=text_inference_request)\n",
    "response.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "798f42a9-2b58-4b69-88c7-dd6b5b6dc7ad",
   "metadata": {},
   "source": [
    "### Create and Deploy a Pipeline connecting our deployed Models"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bd4c5c1-0ad2-48b1-a847-a235bb6e35b4",
   "metadata": {},
   "source": [
    "Now that we have our preprocessor and model deployed, we will chain them together with a pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "5054040c-3e22-4317-86c0-280b129d60c4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "apiVersion: mlops.seldon.io/v1alpha1\n",
      "kind: Pipeline\n",
      "metadata:\n",
      "  name: income-classifier-app\n",
      "spec:\n",
      "  steps:\n",
      "    - name: preprocessor\n",
      "    - name: income-classifier\n",
      "      inputs:\n",
      "      - preprocessor\n",
      "  output:\n",
      "    steps:\n",
      "    - income-classifier\n"
     ]
    }
   ],
   "source": [
    "!cat ../../../samples/quickstart/pipelines/income-classifier-app.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a06a15d6-20c0-40bb-85b4-59d3f9a66486",
   "metadata": {},
   "source": [
    "The yaml defines two steps in a pipeline (the preprocessor and model), mapping the outputs of the preprocessor model (`OUTPUT0`) to the input of the income classification model (`INPUT0`). Seldon Core will leverage Kafka to communicate between models, meaning that all data is streamed and observable in real time."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f92f2e1-6d4f-4d40-9851-4731ddfdf484",
   "metadata": {},
   "source": [
    "```mermaid\n",
    "    graph LR\n",
    "        I[(Input)] --> PP\n",
    "        subgraph Pipeline\n",
    "            PP([Preprocessor]) --> |\" OUTPUT0 → INPUT0 \"| M([Model])\n",
    "        end\n",
    "        M --> O[(Output)]\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "1a6d2cd4-014f-4b1b-a67d-5a3d9d977598",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pipeline.mlops.seldon.io/income-classifier-app created\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f ../../../samples/quickstart/pipelines/income-classifier-app.yaml -n seldon-mesh"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "929b9108-c1d8-4eb8-b96e-f8af3e798afa",
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline_endpoint = f\"http://{MESH_IP}/v2/models/income-classifier-app/infer\"\n",
    "pipeline_headers = {\n",
    "    \"Seldon-Model\": \"income-classifier-app.pipeline\"\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7d4b1af-e7dd-4259-86e5-03acc6abd42b",
   "metadata": {},
   "source": [
    "{% hint style=\"info\" %}\n",
    "You will notice that sending a request to the pipeline is achieved by defining `income-classifier-app.pipeline` as the value for **Seldon-Model** in the headers of the request.\n",
    "{% endhint %}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "ffd19c1e-4dc8-4bf5-984e-dd44046843a1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'model_name': '',\n",
       " 'outputs': [{'data': [0],\n",
       "   'name': 'predict',\n",
       "   'shape': [1, 1],\n",
       "   'datatype': 'INT64',\n",
       "   'parameters': {'content_type': 'np'}}]}"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipeline_response = requests.post(\n",
    "    pipeline_endpoint, json=text_inference_request, headers=pipeline_headers\n",
    ")\n",
    "pipeline_response.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37c2c0da-2ea3-4918-a6ed-66451233c796",
   "metadata": {},
   "source": [
    "Congratulations! You have now deployed a Seldon Pipeline that exposes an endpoint for you ML application 🥳. For more tutorials on how to use Core 2 for various use-cases and requirements, see [here](../../examples/README.md)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4390b87-5bf5-4479-87dd-f538df450fd1",
   "metadata": {},
   "source": [
    "## Clean Up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "9aecad10-5443-436c-8c27-31349824c1bb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pipeline.mlops.seldon.io \"income-classifier-app\" deleted\n",
      "model.mlops.seldon.io \"preprocessor\" deleted\n",
      "model.mlops.seldon.io \"income-classifier\" deleted\n",
      "server.mlops.seldon.io \"mlserver-custom\" deleted\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f ../../../samples/quickstart/pipelines/income-classifier-app.yaml -n seldon-mesh\n",
    "!kubectl delete -f ../../../samples/quickstart/models/preprocessor/preprocessor.yaml -n seldon-mesh\n",
    "!kubectl delete -f ../../../samples/quickstart/models/sklearn-income-classifier.yaml -n seldon-mesh\n",
    "!kubectl delete -f ../../../samples/quickstart/servers/mlserver-custom.yaml -n seldon-mesh"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
